Conventionally, there is a game where a user operates a mobile hand-held terminal (hand-held game apparatus) while holding it, and an event is executed in accordance with the attitude and the position of the mobile hand-held terminal in real space. The mobile hand-held terminal includes a sensor that detects the position and the attitude of the mobile hand-held terminal in real space, and the user of the terminal advances the game by moving the mobile hand-held terminal and changing the attitude of the mobile hand-held terminal. For example, in accordance with the attitude of the mobile hand-held terminal in real space, the mobile hand-held terminal scrolls an image displayed on a display screen of the mobile hand-held terminal. Then, a scope is displayed at the center of the display screen of the mobile hand-held terminal, so that when a predetermined button of the mobile hand-held terminal has been pressed in the state where a virtual object (e.g., an insect object) is included in the scope, it is considered that the virtual object has been caught in the scope.
The mobile hand-held terminal described above is, however, operated by scrolling the image on the basis of the attitude of the mobile hand-held terminal so as to place the virtual object in the scope displayed in a fixed manner at the center of the display screen of the mobile hand-held terminal. Accordingly, the positional relationship between the virtual world and the scope always changes due to the scroll of the image. Thus, the user cannot scroll the image on the basis of the attitude of the mobile hand-held terminal while maintaining the positional relationship between the virtual world and the scope. That is, when attempting to view another direction in the virtual world, the user inevitably moves the scope. This may possibly lead to monotonous operations, and may possibly reduce the verisimilitude and the fun of operations.
Therefore, it is an object of an exemplary embodiment to provide a storage medium having stored thereon an information processing program capable of, when an object that appears in a virtual world and a virtual camera for generating an image of the virtual world are controlled on the basis of the attitude and/or the motion of a display apparatus that displays the image generated by the virtual camera, varying the operations of a user to be performed based on the attitude and/or the motion of the display apparatus, an information processing apparatus, an information processing system, and an information processing method that are capable of, when an object that appears in a virtual world and a virtual camera for generating an image of the virtual world are controlled on the basis of the attitude and/or the motion of a display apparatus that displays the image generated by the virtual camera, varying the operations of a user to be performed based on the attitude and/or the motion of the display apparatus.
To achieve the above object, the exemplary embodiment may employ, for example, the following configurations. It is understood that when the description of the scope of the appended claims is interpreted, the scope should be interpreted only by the description of the scope of the appended claims. If the description of the scope of the appended claims contradicts the description of these columns, the description of the scope of the appended claims has priority.
In an exemplary configuration of a computer-readable storage medium having stored thereon an information processing program according to the exemplary embodiment, the information processing program is executed by a computer of an information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body. The information processing program causes the computer to execute: determining, on the basis of the data output from the portable display apparatus, whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range; controlling, when it has been determined that the direction of the predetermined axis is included in the predetermined range, an action of an object in accordance with the direction of the predetermined axis, the object placed in a virtual world; controlling, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera, for generating an image of the virtual world, in accordance with the direction of the predetermined axis; and displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.
It should be noted that the information processing apparatus may be an apparatus that performs game processing and generates an image based on the game processing, or may be a versatile apparatus such as a general personal computer. The portable display apparatus may have a size small enough to be carried by a user. Typically, the portable display apparatus may be a display apparatus that allows the user to view an image displayed thereon by holding it with both hands. Further, as in a terminal apparatus according to the embodiment described later, the portable display apparatus may or may not include components other than: means for outputting at least data based on the attitude and/or the motion of the portable display apparatus body; and means for displaying the first image.
Based on the above, when a user has performed an operation using a portable display apparatus within a predetermined range, the actions of an object and a first virtual camera are controlled in accordance with the direction of a predetermined axis of the portable display apparatus. Accordingly, when the user has performed an operation using the portable display apparatus within the predetermined range, it is possible to, on the basis of the operation, control not only the direction of the line of sight of the first virtual camera for generating a virtual world to be displayed on the portable display apparatus, but also the action of the object placed in the virtual world. This is suitable for the operation of changing the action of the object and the operation of changing a display range displayed on the LCD 61. On the other hand, when the user has performed an operation using the portable display apparatus outside the predetermined range, only the direction of the line of sight of the first virtual camera is changed in accordance with the attitude and the motion of the portable display apparatus. Therefore, the operation using the portable display apparatus outside the predetermined range is suitable for the operation of changing only the display range displayed on the portable display apparatus while maintaining the action of the object. By thus setting at least one predetermined range, the user can perform various operations on the basis of the attitude and the motion of one device. Further, when the user has performed an operation using the portable display apparatus outside the predetermined range, only the direction of the line of sight of the first virtual camera changes. This enables the user to understand that the operation using the portable display apparatus has deviated to a range in which it is not possible to cause the object to take action, and this makes it possible to imply to the user the range in which the object is caused to take action.
In addition, the determination of whether or not the direction of the predetermined axis is included in the predetermined range may include determining, on the basis of the data output from the portable display apparatus, whether or not the direction of the predetermined axis in a left-right direction in real space is included in a first range set in the left-right direction. In this case, when it has been determined that the direction of the predetermined axis in the left-right direction in real space is included in the first range, the action of the object may be controlled such that on the basis of the direction of the predetermined axis in the left-right direction in real space, the object rotates and/or moves in a left-right direction in the virtual world.
Based on the above, it is possible to set a predetermined range in which the object is caused to take action, by rotating (yawing) the portable display apparatus to the left and right.
In addition, the determination of whether or not the direction of the predetermined axis is included in the predetermined range may include determining, on the basis of the data output from the portable display apparatus, whether or not the direction of the predetermined axis in a left-right direction in real space is included in a first range set in the left-right direction. In this case, when it has been determined that the direction of the predetermined axis in the left-right direction in real space is included in the first range, the action of the object may be controlled such that on the basis of the direction of the predetermined axis in the left-right direction in real space, the object rotates and/or moves in a left-right direction in the virtual world.
Based on the above, it is possible to set the predetermined range in which the object is caused to take action, for each of the case where the portable display apparatus is rotated (yawed) to the left and right, and the case where the portable display apparatus is rotated (pitched) upward and downward.
In addition, the first range may have an angular range of a predetermined size. The second range may have an angular range of a size different from the size of the angular range of the first range.
Based on the above, it is possible to set the predetermined range in which the object is caused to take action, to angular ranges different from each other between the case where the portable display apparatus is rotated (yawed) to the left and right, and the case where the portable display apparatus is rotated (pitched) upward and downward. This makes it possible to set an appropriate range in accordance with an operation environment when the user holds the portable display apparatus and a game environment.
In addition, the first range may be a range having the same angular ranges in a left-right direction about a reference direction in real space. The second range may be a range having angular ranges different from each other in an up-down direction with respect to a horizontal direction in real space.
Based on the above, it is possible to set the center of the predetermined range in which the object is caused to take action, in directions different from each other between the case where the portable display apparatus is rotated (yawed) to the left and right, and the case where the portable display apparatus is rotated (pitched) upward and downward. This makes it possible to set the range at an appropriate position in accordance with an operation environment when the user holds the portable display apparatus and a game environment.
In addition, when it has been determined that the direction of the predetermined axis is included in the predetermined range, an attitude of the first virtual camera may be controlled such that a direction of a line of sight of the first virtual camera is the same as a direction of the object in the virtual world.
Based on the above, if the direction of the predetermined axis of the portable display apparatus is included in the predetermined range, in accordance with a change in the direction of the object, also the direction of the line of sight of the first virtual camera changes. As a result, the direction of the object and the direction of the line of sight of the first virtual camera are controlled on the basis of the attitude and/or the motion of the portable display apparatus. This makes it possible, in accordance with the user directing the portable display apparatus in the direction that they wish to view, to change the direction of the object as well, and provide the user with an image as if peeping at the virtual world through the portable display apparatus. It is also possible to provide the user with a feeling as if being in the virtual world. Further, the direction of the object can be set on the basis of the attitude and/or the motion of the portable display apparatus, and the virtual world viewed in the direction of the object is displayed on the portable display apparatus. This achieves the operation of setting the direction of the object in an intuitive manner. This facilitates setting the direction of the object to the direction desired by the user.
In addition, the action of the object may be controlled on the basis of a change in the direction of the predetermined axis. The action of the first virtual camera may be controlled on the basis of the change in the direction of the predetermined axis.
Based on the above, the user can control the action of the object and the action of the first virtual camera by changing the attitude of and/or the position of the portable display apparatus.
In addition, the action of the object may be controlled such that the greater the change in the direction of the predetermined axis, the greater the object moves. The action of the first virtual camera may be controlled such that the greater the change in the direction of the predetermined axis, the greater the first virtual camera moves.
Based on the above, the user can control the amounts of movement (including the amounts of rotational movement) of the object and the first virtual camera on the basis of the amount of change in the attitude and/or the position of the portable display apparatus.
In addition, the action of the object may be controlled such that the object rotationally moves in accordance with an angle of the predetermined axis. The action of the first virtual camera may be controlled such that the first virtual camera rotationally moves in accordance with the angle of the predetermined axis.
Based on the above, the user can control the action of the object and the action of the first virtual camera on the basis of the angle of the portable display apparatus.
In addition, the information processing program may further cause the computer to execute calculating an attitude and/or a motion of the portable display apparatus on the basis of the data output from the portable display apparatus. In this case, on the basis of the attitude and/or the motion of the portable display apparatus, it may be determined whether or not the direction of the predetermined axis is included in the predetermined range.
Based on the above, it is possible to calculate the attitude and/or the motion of the portable display apparatus using data output from the portable display apparatus. This makes it possible to determine the direction of the predetermined axis, and control the actions of the object and the first virtual camera on the basis of the attitude and/or the motion of the portable display apparatus.
In addition, the attitude and/or the motion of the portable display apparatus may be calculated with respect to a predetermined direction in real space. On the basis of the attitude and/or the motion of the portable display apparatus with respect to the predetermined direction in real space, the action of the object may be controlled with respect to a direction that corresponds to the predetermined direction and is set in the virtual world. On the basis of the attitude and/or the motion of the portable display apparatus with respect to the predetermined direction in real space, the action of the first virtual camera may be controlled with respect to the direction that corresponds to the predetermined direction and is set in the virtual world.
Based on the above, on the basis of the attitude and/or the motion of the portable display apparatus with respect to a predetermined direction in real space, it is possible to control the actions with respect to a predetermined direction that corresponds to the predetermined direction in real space and is included in the virtual world.
In addition, the attitude and/or the motion of the portable display apparatus may be calculated with respect to a direction of gravity in real space, using the direction of gravity as the predetermined direction. On the basis of the attitude and/or the motion of the portable display apparatus with respect to the direction of gravity in real space, the action of the object may be controlled with respect to a direction of gravity set in the virtual world. On the basis of the attitude and/or the motion of the portable display apparatus with respect to the direction of gravity in real space, the action of the first virtual camera may be controlled with respect to the direction of gravity set in the virtual world.
Based on the above, on the basis of the attitude and/or the motion of the portable display apparatus with respect to the direction of gravity, it is possible to control the actions with respect to the same direction of gravity.
In addition, at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space may be calculated. At least a range about the direction of gravity in real space may be set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, it may be determined whether or not the direction of the predetermined axis is included in the predetermined range. On the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, the action of the object may be controlled such that the object rotates about the direction of gravity set in the virtual world. On the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, the action of the first virtual camera may be controlled such that the first virtual camera rotates about the direction of gravity set in the virtual world.
Based on the above, the portable display apparatus may be directed leftward and rightward in real space, whereby it is possible to control the actions of the object and the first virtual camera such that the directions of the object and the first virtual camera are directed leftward and rightward in the virtual world.
In addition, at least the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about a horizontal direction perpendicular to the direction of gravity in real space may be calculated. At least a range about the horizontal direction in real space may be set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, it may be determined whether or not the direction of the predetermined axis is included in the predetermined range. On the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, the action of the object may be controlled such that the object swings upward and downward about a horizontal direction that corresponds to the horizontal direction in real space and is set in the virtual world. On the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, the action of the first virtual camera may be controlled such that the first virtual camera swings upward and downward about the horizontal direction that corresponds to the horizontal direction in real space and is set in the virtual world.
Based on the above, the portable display apparatus may be directed upward and downward in real space, whereby it is possible to control the actions of the object and the first virtual camera such that the directions of the object and the first virtual camera are directed upward and downward in the virtual world.
In addition, at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of a first axis and a second axis that are orthogonal to a perspective direction of, and perpendicular to, a display screen of the portable display apparatus may be calculated, the first image displayed on the display screen. Each of a range about the first axis and a range about the second axis may be set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the first axis and the second axis, it may be determined whether or not the direction of the predetermined axis is included in each predetermined range. When it has been determined that the direction of the predetermined axis is included in the predetermined range about the first axis, the action of the object may be controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the first axis, the object rotates about an axis that corresponds to the first axis and is set in the virtual world. When it has been determined that the direction of the predetermined axis is included in the predetermined range about the second axis, the action of the object may be controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the second axis, the object rotates about an axis that corresponds to the second axis and is set in the virtual world. The action of the first virtual camera may be controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the first axis and the second axis, the first virtual camera rotates about each of axes that correspond to the first axis and the second axis and are orthogonal to a direction of a line of sight of the first virtual camera.
Based on the above, the portable display apparatus may be moved so as to rotate about two axes orthogonal to the perspective direction of a display screen of the portable display apparatus in real space, whereby it is possible to control the actions of the object and the first virtual camera such that the directions of the object and the first virtual camera are directed upward, downward, leftward, and rightward.
In addition, at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the display apparatus about each of an axis along a width direction of the display screen and an axis along a height direction of the display screen may be calculated, each axis being orthogonal to the perspective direction. Each of a range about the axis along the width direction and a range about the axis along the height direction may be set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the axis along the width direction and the axis along the height direction, it is determined whether or not the direction of the predetermined axis is included in each predetermined range. When it has been determined that the direction of the predetermined axis is included in the predetermined range about the axis along the width direction, the action of the object may be controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the width direction, the object rotates about a horizontal axis that corresponds to the axis along the width direction and is set in the virtual world. When it has been determined that the direction of the predetermined axis is included in the predetermined range about the axis along the height direction, the action of the object may be controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by the portable display apparatus about the axis along the height direction, the object rotates about a vertical axis that corresponds to the axis along the height direction and is set in the virtual world. The action of the first virtual camera may be controlled such that: in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the width direction, the first virtual camera rotates about a horizontal axis that is orthogonal to a direction of a line of sight of the first virtual camera and is included in the virtual world; and in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the height direction, the first virtual camera rotates about a vertical axis in the virtual world.
Based on the above, the portable display apparatus may be moved so as to rotate about the height direction and the width direction of the display screen of the portable display apparatus in real space, whereby it is possible to control the actions of the object and the first virtual camera such that the directions of the object and the first virtual camera are directed upward, downward, leftward, and rightward.
In addition, the calculation of the attitude and/or the motion of the portable display apparatus may include: setting a reference direction of the portable display apparatus in real space; and calculating, on the basis of the data output from the portable display apparatus, a difference in left-right angle about a vertical direction in real space between a current direction of the predetermined axis and the reference direction, and a difference in up-down angle about a horizontal direction in real space between the current direction of the predetermined axis and the horizontal direction. The determination of whether or not the direction of the predetermined axis is included in the predetermined range may include: determining whether or not the difference in left-right angle is included in a first range set about the vertical direction in real space; and determining whether or not the difference in up-down angle is included in a second range set about the horizontal direction in real space. In this case, when it has been determined that the difference in left-right angle is included in the first range, the action of the object may be controlled such that the object rotates and/or moves in a left-right direction in the virtual world on the basis of the difference in left-right angle. When it has been determined that the difference in up-down angle is included in the second range, the action of the object may be controlled such that the object rotates and/or moves in an up-down direction in the virtual world on the basis of the difference in up-down angle. The action of the first virtual camera may be controlled such that: the first virtual camera rotates and/or moves in a left-right direction in the virtual world on the basis of the difference in left-right angle; and the first virtual camera rotates and/or moves, on the basis of the difference in up-down angle, in an up-down direction and about a left-right direction that is orthogonal to a direction of a line of sight of the first virtual camera and is included in the virtual world.
Based on the above, it is possible to control the actions of the object and the first virtual camera on the basis of reference directions and the horizontal directions that are set in real space and the virtual world.
In addition, image data indicating the first image may be output to the portable display apparatus. The portable display apparatus may include an image data acquisition unit and a display unit. The image data acquisition unit acquires the image data output from the information processing apparatus. The display unit displays the first image indicated by the image data acquired by the image data acquisition unit.
Based on the above, the portable display apparatus can function as a so-called thin-client terminal, which does not perform information processing such as game processing.
In addition, the information processing program further causing the computer to execute generating compression image data by compressing the image data indicating the first image. In this case, the generated compression image data may be output to the portable display apparatus. The image data acquisition unit may acquire the compression image data output from the information processing apparatus. The portable display apparatus may further include a display image decompression unit. The display image decompression unit decompresses the compression image data to obtain the image data indicating the first image. In this case, the display unit displays the first image indicated by the image data that has been acquired by the image data acquisition unit and has been decompressed by the display image decompression unit.
Based on the above, the first image is decompressed before being output from the information processing apparatus to the portable display apparatus. This makes it possible to output the first image at a high speed, and reduce delay caused between the generation of the first image and the display of the first image on the portable display apparatus.
In addition, besides the first image, a second image representing the virtual world viewed from a second virtual camera may be further displayed on another display apparatus connected to the information processing apparatus.
It should be noted that said another display apparatus described above is a display apparatus connected to the information processing apparatus, like a monitor 2 according to the embodiment described later. Said another display apparatus may be a component separate from the portable display apparatus, and may be any apparatus so long as it is capable of displaying the second image generated by the information processing apparatus. For example, said another display apparatus described above may be integrated with the game apparatus (in a single housing).
Based on the above, when processing based on the operation of moving and changing the attitude of the portable display apparatus is performed, it is possible to display the results of the processing not only on the portable display apparatus but also on said another display apparatus connected to the information processing apparatus. This enables the user to use, in accordance with the state of the operation or the user's preference, either one of images displayed on, for example, two apparatuses, and also view an image suitable for an operation of the user. Further, it is possible to use an image displayed on said another display apparatus connected to the information processing apparatus, as, for example, an image to be viewed by another person different from the user. This makes it possible to provide a viewing environment suitable also for the case where a plurality of people view the results of the processing.
In addition, the information processing program may further cause the computer to execute generating compression image data by compressing the image data indicating the first image. In this case, the generated compression image data may be output to the portable display apparatus, and, besides the compression image data, image data indicating the second image may be output to said another display apparatus without being compressed. The portable display apparatus includes an image data acquisition unit, a display image decompression unit, and a display unit. The image data acquisition unit acquires the compression image data output from the information processing apparatus. The display image decompression unit decompresses the compression image data to obtain the image data indicating the first image. The display unit displays the first image indicated by the image data that has been acquired by the image data acquisition unit and has been decompressed by the display image decompression unit.
Based on the above, the first image is decompressed and then output from the information processing apparatus to the portable display apparatus. This makes it possible to output the first image at a high speed, and reduce delay caused between the generation of the first image and the display of the first image on the portable display apparatus.
In addition, the information processing program may further cause the computer to execute setting, on the basis of a position of the object in the virtual world, the second virtual camera, for generating an image of the virtual world, at a position different from a position of the first virtual camera such that the object is included in the second image.
Based on the above, the same virtual world is displayed not only on the portable display apparatus but also on said another display apparatus, and images of the virtual world that are different in the point of view are displayed thereon. This enables the user to use, in accordance with the state of the operation or the user's preference, either one of the images displayed on the two apparatuses when performing an operation.
In addition, the second virtual camera may be set at a position further away from the object than the first virtual camera is from the object. A range wider than a range of the virtual world represented by the first image may be displayed as the second image on said another display apparatus.
Based on the above, an image of the virtual world in a display range wider than that of an image of the virtual world displayed on the portable display apparatus is displayed on said another display apparatus connected to the information processing apparatus. This makes it possible to display on each display apparatus, for example, an image suitable for an operation of the user when the state of the virtual world is presented to the user.
In addition, the second virtual camera may be set at a position of viewing the object from a bird's-eye view in the virtual world. An image obtained by viewing from a bird's-eye view the object placed in the virtual world may be displayed as the second image on said another display apparatus.
Based on the above, an image of the virtual world based on the attitude and/or the motion of the portable display apparatus is displayed on the portable display apparatus, and an image of the virtual world obtained by looking down upon it is displayed on another display apparatus connected to the information processing apparatus. This makes it possible to display on each display apparatus, for example, an image suitable for an operation of the user when the state of the virtual world is presented to the user.
In addition, the portable display apparatus may include at least one of a gyro sensor and an acceleration sensor. On the basis of data output from the at least one of the gyro sensor and the acceleration sensor, it may be determined whether or not the direction of the predetermined axis is included in the predetermined range.
Based on the above, using the data that is output from the gyro sensor and indicates the angular velocity generated in the portable display apparatus and/or the data that is output from the acceleration sensor and indicates the acceleration generated in the portable display apparatus, it is possible to accurately calculate the attitude and the motion of the portable display apparatus.
In addition, the exemplary embodiment may be carried out in the forms of an information processing apparatus and an information processing system, each including units that perform the above processes, and an information processing method including the above operations.
The exemplary embodiment enables the user to perform various operations on the basis of the attitude and the motion of one device, by setting at least one predetermined range.
These and other objects, features, aspects and advantages of the exemplary embodiment will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.